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DETAILED ACTION 
Continued Examination Under 37 CFR 1.114 

1 . A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 
6/8/2006 has been entered. 

Response to Amendment 

2. The amendment was received on June 8,2006. Claims 1-26 are pending. 

Claim Objections 

3. Due to the amendment, the objection to claim 10 is withdrawn. 

Response to Arguments 

4. Applicant's arguments on page 13, lines 6-8 filed 6/8/2006 have been fully 
considered but they are not persuasive and states: 



"... Lelong fails to teach. ..that the tangible object plane being defined by a spatial 
orientation of the tangible object..." 



Application/Control Number: 09/940,432 Page 3 

Art Unit: 2624 

However, the examiner respectfully disagrees since Lelong et al. (US Patent 
5,444,478 A) discloses that the tangible object plane (fig. 5A, label: "M") being defined 
by a spatial orientation (of either the coordinate system (yO.Xo.Zo) of an image lo or the 
coordinate system (Y,Z,X,Zo) of a camera, "P" as shown in fig. 5A) of the tangible 
object. 

5. Applicant's arguments on page 13, lines 16-18, filed 6/8/2006 have been fully 
considered but they are not persuasive and states: 

"... Lelong fails to teach. ..the generating of a distortion-corrected image on a 
projection plane by projecting the standard image onto the projection plane 
based on the direction of the tangible object plane. .. " 

However, the examiner respectfully disagrees since Lelong et al. discloses the 
generating of a distortion-corrected image (as shown in figures 7A-7C) on a projection 
plane (as shown in fig. 7C) by projecting the standard image (fig. 7A) onto the projection 
plane based on the direction (as shown in fig. 5A that describes a coordinate system 
where one of ordinary skill in the art can use coordinates as directions to find point "M" 
in fig. 5A. Note that the claimed direction is broad enough that other interpretations are 
possible. Such as "ray PM" in col. 13, line 61 where ray is interpreted as a pointer to 
point M in fig. 5A. Upon further review of the specification, especially page 36 with 
respect to "n", the claimed direction appears to be a normal of a tangible object. If that is 
correct then a distinction is made in Lelong et al. Referring to fig. 1 A, Lelong et al. 



Application/Control Number: 09/940,432 Page 4 

Art Unit: 2624 

shows 90 degree symbols that corresponds to the claimed normal, but the 90 degrees 
symbols are with respect to the center of the imaging plane and not a tangible plane of 
the tangible object which is shown in fig. 1 A, label: "M" that has no normal at the 
tangible plane of M. Fig. 1C shows the imaging plane with "m" that represents the 
tangible plane M where m of the imaging plane has no normal at m; a normal appears 
to be at the center of the imaging plane and not at the center of the tangible object 
plane. The same applies for fig. 5A and 5B. Fig. 5B shown ray PM that intersects 
images l 0 and lj where at the intersections of the ray there is no angle or 90 degree 
angle or normal associated with the intersections.) of the tangible object plane. 
6. Applicant's arguments on page 14, lines 3-5, filed 6/8/2006 have been fully 
considered but they are not persuasive and states: 

"... Lelong fails to teach or suggest that an image is automatically selected as the 
standard image based on a calculated direction of the object plane for each of 
the partially overlapping image." 

However, the examiner respectfully disagrees since Lelong et al. discloses that 
an image is automatically selected (via fig. 6, numerals 2 and 205 where numerals 2 
"controlled automatically" in col. 17, line 14) as the standard image based on a 
calculated direction (as shown in fig. 5A that describes a coordinate system where one 
of ordinary skill in the art can use coordinates as directions to find point "M" in fig. 5A ) 
of the object plane for each of the partially overlapping images. 
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7. Applicant's arguments, see page 14,lines 5,6, filed 6/8/2006, with respect to 
1 02(e) have been fully considered and are persuasive and states: 

"Lelong never calculates an object plane, but merely uses a virtual plane lo of a 
virtual camera view Co " 

Upon further review of the specification, this statement is interpreted to mean: 

"Lelong never calculates an object plane [at the tangible object plane], but merely 
uses a virtual plane lo of a virtual camera view Co ." 

Upon further review of the specification, a tangible object plane or tangible 
surface is calculated as shown in fig. 12, label PL via equation (7) and in figure 36,label: 
"PL" which corresponds to the claimed object plane. Lelong et al. calculates an imaging 
plane or virtual plane, Co, and is silent with calculating a plane at the site of the tangible 
plane. Instead, Lelong et al. calculates the ray, PM, and the point, M, at the site of the 
tangible object plane. Lelong et al. is not too clear as to whether a plurality of points 
form a plane where a normal can be computed from the plane of points at the site of the 
tangible plane. 

8. Applicant's arguments with respect to claim 26 on page 14 filed 6/8/2006 have 
been fully considered but they are not persuasive. See the rejection of claim 26, below. 
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9. Applicants arguments on page 15, lines 2-5, filed 6/8/2006 have been fully 
considered but they are not persuasive and states: 

"Lee.. .does not teach. ..the calculation of a direction of the tangible object plane, 
the tangible object plane being defined by a spatial orientation of the tangible 
object." 

However, the examiner respectfully disagrees since Lee (US Patent 6,507,366 
B1 ) teaches the calculation of a direction of the tangible object plane (via the coordinate 
system of fig. 4 that provides coordinates that provide directions in the X,Y, and Z 
directions to find a plane, IMAGE PLANE (tt) of fig. 4, of a tangible object, OBJECT in 
fig. 4), the tangible object plane being defined by a spatial orientation (or coordinate 
system (X,Y,Z) of fig. 4) of the tangible object. 

Upon further review of the specification and in another interpretation, Lee does 
not teach the calculation of a normal at (emphasis added) the tangible object plane, the 
tangible object plane being defined by a spatial orientation of the tangible object." 
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Claim Rejections - 35 USC § 102 

10. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 

1 1 . Claims 1 ,3-10 and 12-26 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Lelong et al. (US Patent 5,444,478 A). 

Regarding claims 1 and 9, Lelong et al. discloses an image processing method 
and apparatus for correcting image distortions caused by oblique imaging in which a 
tangible object on an object plane is captured from different oblique directions to obtain 
a plurality of partially overlapping images, comprising the steps of: 

a) a correspondence detecting unit (Fig. 3,num. 200 is used in a "calibration" 
in col. 16, line 63 via the method from col. 16, line 63 to col. 17, line 13) determining a 
feature point ("seam" in col. 16, line 47 and shown in fig. 1 E, label: "L 0 ") of one of the 
plurality of partially overlapping images (in an "overlap zone" in col. 16, lines 49,50) 
corresponding to a common location of the tangible object (The building of figures 7A 
and 7B), shared by the plurality of partially overlapping images, and 
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b) determining a matched point of one of the other partially overlapping 
images corresponding to the feature point (to create "a perfect juxtapo-sition" in col. 16, 
lines 66,67) so that a direction (Fig. 5B ( direction or line "PM".) of the tangible object 
plane (Fig. 5B, label" l 0 ") is calculated based on the feature point and the matched point 
(Note that the above mentioned perfect juxtaposition is interpreted as matching features 
of an image Si in fig. 7C to features of an image Sj in fig. 7C along a seam, L in fig. 7C 
to create a seamless image as shown in fig. 7D where no seam is visible and respective 
portions of the image such as the building and trees are matched to create "a 
uniform.. .image" in col. 17, lines 12,13.), the tangible object plane (fig. 5A, label: "M") 
being defined by a spatial orientation (of either the coordinate system (yO,Xo,Zo) of an 
image lo or the coordinate system (Y,Z,X,Zo) of a camera, "P" as shown in fig. 5A) of the 
tangible object; 

c) a standard image setting unit (fig. 3,num. 205) selecting one ("selecting 
one" in col. 6, line 46) of the plurality of partially overlapping images (Fig. 6, num. 205) 
as a standard image (via a "reference... camera" in col. 9, line 66 that creates a "source 
image" in col. 9, line 67 which is interpreted as the claimed standard image) whose 
image distortions are to be corrected (as shown in fig. 7D relative to fig. 7C); and 

d) a distortion correcting unit (fig. 3, num. 112) generating a distortion- 
corrected image (fig. 7D) on a projection plane (or "projected in the... plane" in col. 8, 
line 1 1) by projecting the standard image onto the projection plane based on the 
direction of the tangible object plane such that image distortions in the standard image 
are eliminated. 
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Regarding claim 3, Lelong et al. discloses the image processing method 
according to claim 1 wherein in said selecting step, one of the plurality of partially 
overlapping images is automatically ("automatically" in col. 17, line 14) selected as the 
standard based on a direction of a straight-line pattern (fig. 1E, label: "L 0 ") contained in 
each image. 

Claims 4 and 5 are rejected the same as claim 1. Thus, argument similar to that 
presented above for claim 1 is equally applicable to claims 4 and 5. 

Claim 6 is rejected the same as claim 1 . Thus, argument similar to that 
presented above for claim 1 is equally applicable to claim 6 except for the remaining 
limitations of: 

a) selecting one of the plurality of partially overlapping images as a standard 
image (Fig. 6, num. 205) that contains a smallest amount of image distortions (via a 
"feedback" in col. 16, line 64 to create "a target image without any faults" in col. 16, lines 
67,68. Thus, if a target image is created without any faults then the images that were 
used to create the target image also have no faults.) among the plurality of partially 
overlapping images; and 

b) combining the other partially overlapping images, which are projected (as 
shown in fig. 1 A via a plurality of lines that radiate from point "P".) onto an image 
surface (Fig. 1 A, label: "l 0 ") of the standard image with respect to each of the other 
partially overlapping images, so that a composite image (fig. 7D) is generated on the 
image surface so as to correct image distortions in the standard image (relative to the 
image of fig. 7C). 
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Claim 7 is rejected the same as claims 1 and 9. Thus, argument similar to that 
presented above for claims 1 and 9 is equally applicable to claim 7 except for the 
limitation disclosed in Lelong et al. of: 

a) an image composition unit ("Devices capable of realiz-ing these 
operations" in col. 17, lines 5,6 such as "perfect juxtapo-sition" in col. 16, lines 66,67.) 
combining (via a "joint" in col. 9, line 42) the other partially overlapping images, which 
are projected (as shown in fig. 1 A via a plurality of lines that radiate from point "P".) onto 
an image surface (Fig. 1 A, label: 'V) of the standard image with respect to each of the 
other partially overlapping images, so that a composite image (fig. 7D) is generated on 
the image surface so as to correct image distortions in the standard image (relative to 
the image of fig. 7C). 

Regarding claim 8, Lelong et al. discloses the image processing apparatus 
according to claim 7, 

a) wherein said standard image setting unit (fig. 3, num. 205) is configured 
such that a user (via fig. 3, num. 2) is required to select the standard image when taking 
the original image from one of the oblique directions (Fig. 7A shows an image of a 
scene at one angle and fig. 7B shows another image of the same scene at another 
angle so that when both images are combined as shown in fig. 7C distortions are shown 
because the images of the same scene were taken at different angles.), and 

b) wherein said image processing apparatus further comprises: 

b1 ) a notification unit (Fig. 21 ,num. 2) which notifies the user that the 
standard image is currently taken (via fig. 3, num. 205). 
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Regarding claim 10, Lelong et al. discloses the image processing apparatus 
according to claim 9, further comprising: 

a) a plurality of imaging units (Fig. 1G, label "P") which respectively input the 
plurality of partially overlapping images that are generated by taking the original image 
from the oblique directions. 

Claims 12-14 are rejected the same as claims 3-5, respectively. Thus, argument 
similar to that presented above for claims 3-5 is equally applicable to claims 12-14, 
respectively. 

Claim 15 is rejected the same as claims 1 and 9. Thus, argument similar to that 
presented above for claims 1 and 9 is equally applicable to claim 15 except for the 
additional limitation of a computer-readable storage medium as disclosed in Lelong et 
al. in fig. 3, num, 210. 

Claim 16 is rejected the same as claims 7 and 15. Thus, argument similar to that 
presented above for claims 7 and 15 is equally applicable to claim 16. 

Regarding claim 17, Lelong et al. discloses the image processing method of 
claim 1 , wherein said standard image is projected with a perspective projection matrix 
operation (or " 'perspective transform'" in col. 14, line 57). 

Claims 18-22 are rejected the same as claim 17. Thus, argument similar to that 
presented above for claim 17 is equally applicable to claims 18-22. 
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Regarding claim 23, Lelong discloses the image processing method of claim 17, 
wherein said perspective projection matrix is calculated based on coordinates (or 
"coordinates" in col. 14, line 61) of at least four combinations of feature points of the 
standard image and matched points corresponding thereto (via a "4 X 4 matrix" in col. 
14, line 62). 

Regarding claim 24 see "least-squares method" in col. 15, line 17. 

Regarding claim 25, Lelong et al. discloses the image processing method 
according to claim 1 , wherein the tangible object is an image on a planar surface (or a 
"scene" in col. 2, line 27 that corresponds to the claimed image that has "surface area" 
in col. 2, line 27. 

12. Claim 1 is rejected under 35 U.S.C. 102(e) as being anticipated by Cahill et al. 
(US Patent 6,507,665 B1). 

Regarding claims 1 , Cahill et al. discloses an image processing method 
for correcting image distortions caused by oblique imaging in which a tangible object on 
an object plane is captured from different oblique directions to obtain a plurality of 
partially overlapping images, comprising the steps of: 

a) determining a feature point (fig. 3,num. 13) of one of the plurality of 
partially overlapping images (fig. 3, numerals 13 and 16) corresponding to a common 
location (fig. 3,num. 18) of the tangible object, shared by the plurality of partially 
overlapping images, and 
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b) determining a matched point (fig. 3, num. 15) of one of the other partially 
overlapping images corresponding to the feature point so that a direction (fig. 3, num. 
17) of the tangible object plane is calculated based on the feature point and the 
matched point, the tangible object plane being defined by a spatial orientation (as 
shown in fig. 3 that shows a space of points) of the tangible object; 

c) selecting one of the plurality of partially overlapping images (either image 
of fig. 3,numerals 14 and 16 that is represented in fig. 5B,num. 36) as a standard image 
whose image distortions (that are "inherent" in col. 7, line 54) are to be corrected (via 
fig. 2, num. 8); and 

d) generating a distortion-corrected image (fig. 5B t num. 36) on a projection 
plane (or cylinder) by projecting the standard image onto the projection plane based on 
the direction of the tangible object plane (as determined in fig. 2, num. 5) such that 
image distortions in the standard image are eliminated. 



Application/Control Number: 09/940,432 Page 14 

Art Unit: 2624 

Claim Rejections - 35 USC § 103 

1 3. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

14. Claims 2 and 1 1 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Lelong et al. (US Patent 5,444,478 A) in view of Lee (US Patent 6,507,366 B1). 

Regarding claim 2, Lelong et al. teaches the image processing method according 
to claim 1 wherein in said selecting step, one of the plurality of partially overlapping 
images is automatically (or "automatically" in col. 17, line 14) selected (via fig. 3, num. 2 
which can be an automatic process that selects the standard image via fig. 3, num. 205.) 
as the standard image. 

Lelong et al. does not teach the remaining limitation of selecting the standard 
image based on a ratio of an area of an object region to an entire area of each image, 
but does teach an "automatic function" in col. 4, line 61 for "monitor[ing]" in col. 4, line 
40. Thus, Lelong et al. suggests that an automatic process can be used for monitoring 
automatically, but does not provide enough details on the automatic aspect of 
monitoring. Thus, one of ordinary skill in the art will be motivated to find an automatic 
process of monitoring so that a person is freed from monitoring a scene and so that the 
person can perform other duties. 
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Lee teaches "monitoring" in col. 1, line 12 and "automatically tracking" in the 
abstract as taught in Lelong et al. and the remaining limitation of claim 2: 

a) selecting (via fig. 3A,num. 360) based on a ratio of an area of an object 
region (or "region ratio" in col. 9, line 1) to an entire area (or "effective region" in col. 9, 
line 3) of each image. 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to modify Lelong et al.'s teaching of selecting the standard image and 
automatic monitoring with Lee's teaching of selecting or automatic tracking via fig. 
3A,num. 360 so that Lelong et al's selecting the standard image, either one of lj shown 
twice as shown in fig. 1 E of Lelong et al. can be automatically tracked or monitored 
freeing a person from monitoring to perform other duties. 

Claim 1 1 is rejected the same as claim 2. Thus, argument similar to that 
presented above for claim 2 is equally applicable to claim 1 1 . 

15. Claim 26 is rejected under 35 U.S.C. 103(a) as being unpatentable over Lelong 
et al. (US Patent 5,444,478 A) in view of Honma (US Patent 6,304,31 3 B1 ). 

Regarding claim 26, Lelong et al. does not teach the limitations of claim 26 of a 
smallest inclination angle of a viewing direction of the oblique imaging, but 
does teach that "All the means for correct-ing the distortions of the objectives may be 
known to those skilled in the art." in col. 14, lines 4,5. Thus, Lelong et al. suggests to 
one of ordinary skill in the art that any means can be used to correct distortions of an 
objective. 
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Honma teaches a means of correcting a distortion using "perspective correction" 
in col. 13, line 20 of an objective or "optical lens" in col. 13, line 20 and the remaining 
limitation of: 

a smallest inclination angle (or "correctly face[d]" in col. 1 1 , line 2) of a viewing 
direction (of a "camera" in col. 1 1 , line 1 .). 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to modify Lelong et al.'s teaching of a means to correct distortions of an 
objective with Honma's teaching of perspective correction, because Honma's teaching 
provides a "[simple] lens" in col. 13, line 21 with "less cost than conventional devices." in 
col. 13, line 25. 

In addition, Honma's teaching of when correctly facing a camera, "perspec-tive 
correction is unnecessary" in col. 11, lines 3,4. Thus, extra time is saved by bypassing 
the step of perspective correction when the camera is correctly faced. 

Conclusion 

16. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

Yano et al. (US Patent 6,590,704 B1) is pertinent as teaching a method of 
correcting distortion of two views as shown in fig. 28A-28C. 

Myers et al. (US Patent 6,278,460 B1 ) is pertinent as teaching a method of 
matching points in fig. 17, num. 1713 for later ray casting which is interpreted to be a 
direction that calculates a 3-D model. 



Application/Control Number: 09/940,432 



Page 17 



Art Unit: 2624 

Katayama et al. (US Patent 5,668,595 A) is pertinent as teaching a method of 
keystone correction of two matching points, fig. 6, labels: Al and Ar, that correspond to 
a common point A of fig. 6. 

17. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Dennis Rosario whose telephone number is (571) 272- 
7397. The examiner can normally be reached on 9-5. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Bhavesh Mehta can be reached on (571) 272-7453. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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